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ABSTRACT 

The decrease in number density of Lyman-a clouds near the background quasar 
is an observational result which is often called the 'proximity' or 'inverse' effect. It is 
thought that, for nearby clouds, the quasar's flux dominates the background radiation 
field, increasing the ionization state of the clouds and reducing the (observed) H I 
column density. 

In this paper we analyse a sample of 11 quasars from the literature for which 
accurate column density estimates of the Lyman-a lines exist. We confirm, to a sig- 
nificance level of more than 3 standard deviations, that the proximity effect exists. If 
it is related to the background flux then the intensity and evolution of the background 
have been constrained. 

Using a maximum likelihood method, we determine the strength of the extragalac- 
tic ionizing background for 2.0 < z < 4.5, taking account of possible systematic errors 
in our determination and estimating the effect of biases inherent in the data. If the 
background is constant we find that it has an intensity of 100 ^{j J23, where J23 is 
defined as 10~ 23 ergs cm -2 Hz -1 sr _1 . There is no significant evidence for a change 
in this value with redshift. 
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1 INTRODUCTION 

1.1 The Lyman— alpha forest 

Spectroscopic observations towards quasars show a large 
number of intervening absorption systems. This 'forest' of 
lines is numerically dominated by systems showing only the 
Lyman-alpha transition — these absorbers are called Ly- 
man alpha clouds. 

Earlier work suggests that the clouds are large, highly 
ionized structures, either pressure confined (eg. Ostriker & 
Ikeuchi 1983) or within cold dark matter structures (eg. 
Miralda-Escude & Rees 1993; Cen, Miralda-Escude & Os- 
triker 1994; Petitjean & Miicket 1995; Zhang, Anninos & 
Norman 1995). However, alternative models do exist: cold, 
pressure confined clouds (eg. Barcons & Fabian 1987, but 
see Rauch et al. 1993); various shock mechanisms (Vishniac 
& Bust 1987, Hogan 1987). 

Low and medium resolution spectroscopic studies of the 
forest generally measure the redshift and equivalent width 
of each cloud. At higher resolutions it is possible to measure 
the redshift (2), H I column density (N, atoms per cm 2 ) and 
Doppler parameter (b, kms -1 ). These are obtained by fitting 
a Voigt profile to the data (Rybicki & Lightman 1979). 



Using a list of N, z and b measurements, and their asso- 
ciated error estimates, the number density of the population 
can be studied. Most work has assumed that the density is 
a separable function of z, N and 6 (Rauch et al. 1993). 

There is a local decrease in cloud numbers near the 
background quasar which is normally attributed to the addi- 
tional ionizing flux in that region. While this may not be the 
only reason for the depletion (the environment near quasars 
may be different in other respects; the cloud redshifts may 
reflect systematic motions) it is expected for the standard 
physical models wherever the ionising flux from the quasar 
exceeds, or is comparable to, the background. 

Since the generally accepted cloud models are both op- 
tically thin to ionizing radiation and highly ionized, it is 
possible to correct column densities from the observed val- 
ues to those that would be seen if the quasar were more 
remote. The simplest correction assumes that the shape of 
the two incident spectra — quasar and background — are 
similar. In this case the column density of the neutral frac- 
tion is inversely proportional to the incident ionizing flux. 

If the flux from the quasar is known, and the depletion 
of clouds is measured from observations, the background flux 
can be determined. By observing absorption towards quasars 
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at different redshifts the evolution of the flux can be mea- 
sured. Bechtold (1993) summarises earlier measurements of 
the ionising flux, both locally and at higher redshifts. 

Recently Loeb & Eisenstein (1995) have suggested that 
enhanced clustering near quasars causes this approach to 
overestimate the background flux. If this is the case then 
an analysis which can also study the evolution of the effect 
gives important information. In particular, a decrease in the 
inferred flux might be expected after the redshift where the 
quasar population appears to decrease. However, if the pos- 
tulated clustering enhancement is related to the turn-on of 
quasars at high redshift, it may conspire to mask any change 
in the ionizing background. 

Section ti describes the model of the population density 
in more detail, including the corrections to flux and redshift 
that are necessary for a reliable result. The data used are 
described in section [| In section [| the quality of the fit is 
assessed and the procedure used to calculate errors in the 
derived parameters is explained. Results are given in section 
|H] and their implications discussed in section ^. Section ^ 
concludes the paper. 



2 THE MODEL 

2.1 Population Density 

The Doppler parameter distribution is not included in the 
model since it is not needed to determine the ionizing back- 
ground from the proximity effect. The model here assumes 
that N and z are uncorrelated. While this is unlikely (Car- 
swell 1995), it should be a good approximation over the re- 
stricted range of column densities considered here. 

The model of the population without Doppler parame- 
ters or the correction for the proximity effect is 



dn(N',z) = A'{l + Z y' (N') 



-a c(i + z) 

H {l + 2q z)i 



dN' dz (1) 



where Ho is the Hubble parameter, qo is the cosmological 
deceleration parameter and c is the speed of light. Correcting 
for the ionizing flux and changing from 'original' (N') to 
'observed' (N) column densities, gives 



dn(N,z)= A{l + zY' (£-) 



c(l + z) 



dN 



H (l + 2q Q z)i A F 



where 
N = N'A F 
f E 

A J" 



dz (2) 

(3) 
(4) 



and f„ is the background flux, f„ is the flux from the quasar 
(4irJ v (z)). 

The background flux J® may vary with redshift. Here 
it is parameterised as a constant, a power law, or two power 
laws with a break which is fixed at zb = 3.25 (the mid- 
point of the available data range). An attempt was made to 
fit models with zb as a free parameter, but the models were 
too poorly constrained by the data to be useful. 

J v (z) = W h 25 model B (5) 



Mz) = 10 



J v (z) = 10 Ja 25 



' 1 + z ' 
. 1 + / 



+ 3.25, 
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Z > ZB 



C 

D & E 



(6) 
(7) 



A large amount of information (figure |3|) is used to con- 
strain the model parameters. The high-resolution line lists 
give the column density and redshift, with associated errors, 
for each line. To calculate the background ionising flux the 
quasar luminosity and redshift must be known (table ^|). Fi- 
nally, each set of lines must have observational completeness 
limits (table H) . 



2.2 Malmquist Bias and Line Blending 

Malmquist bias is a common problem when fitting models 
to a population which increases rapidly at some point (often 
near an observational limit). Errors during the observations 
scatter lines away from the more populated regions of pa- 
rameter space and into less populated areas. Line blending 
occurs when, especially at high redshifts, nearby, overlap- 
ping lines cannot be individually resolved. This is a conse- 
quence of the natural line width of the clouds and cannot be 
corrected with improved spectrographic resolution. The end 
result is that weaker lines are not detected in the resultant 
'blend'. Both these effects mean that the observed popula- 
tion is not identical to the 'underlying' or 'real' distribution. 



2.2.1 The Idea of Data Quality 

To calculate a correction for Malmquist bias we need to un- 
derstand the significance of the error estimate since any cor- 
rection involves understanding what would happen if the 
'same' error occurs for different column density clouds. The 
same physical cloud cannot be observed with completely dif- 
ferent parameters, but the same combination of all the com- 
plex factors which influence the errors might affect a line 
with different parameters in a predictable way. If this idea 
of the 'quality' of an observation could be quantified it would 
be possible to correct for Malmquist bias: rather than the 
'underlying' population, one reflecting the quality of the ob- 
servation (ie. including the bias due to observational errors) 
could be fitted to the data. 

For example, if the 'quality' of an observation was such 
that, whatever the actual column density measured, the er- 
ror in column density was the same, then it would be triv- 
ial to convolve the 'underlying' model with a Gaussian of 
the correct width to arrive at an 'observed' model. Fitting 
the latter to the data would give parameters unaffected by 
Malmquist bias. Another example is the case of galaxy mag- 
nitudes. The error in a measured magnitude is a fairly sim- 
ple function of source brightness, exposure time, etc., and 
so it is possible to correct a flux-limited galaxy survey for 
Malmquist bias. 



2.2.2 Using Errors as a Measure of Quality 

It may be possible to describe the quality of a spectrum by 
the signal to noise level in each bin. From this one could, for 
a given line, calculate the expected error in the equivalent 
width. The error in the equivalent width might translate, 
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Figure 1. The distribution of errors in column density. 



depending on whether the absorption line was in the linear 
or logarithmic portion of the 'curve of growth', to a normal 
error in either N or log(iV). But in this idealised case it has 
been assumed that the spectrum has not been re-binned, 
leaving the errors uncorrelated; that the effect of overlap- 
ping, blended lines is unimportant; that there is a sudden 
transition from a linear to logarithmic curve of growth; that 
the resulting error is well described by a normal distribu- 
tion. None of this is likely to be correct and, in any case, 
the resulting analysis, with different 'observed' populations 
for every line, would be too unwieldy to implement, given 
current computing facilities. 

A more pragmatic approach might be possible. A plot 
of the distribution of errors with column density (figure |l]) 
suggests that the errors in log(iV) are of a similar magni- 
tude for a wide range of lines (although there is a significant 
correlation between the two parameters). Could the error in 
log(iV) be a sufficiently good indicator of the 'quality' of an 
observation? 

If the number density of the underlying population is 
n'(N) dlog(JV) then the observed population density for a 
line with error in log(AT) of crjv is: 



n(N) dlog(AT) oc / n (N10 x ) 



exp 



2*4 



da; 



(8) 



For a power law distribution this can be calculated analyti- 
cally and gives an increased probability of seeing lines with 
larger errors, as expected. For an underlying population den- 
sity N'* 3 dN the increase is exp ((1 - /3) 2 (a N log 10) 2 /2). 

This gives a lower statistical weight to lines with larger 
errors when fitting. For this case — a power law and log- 
normal errors — the weighting is not a function of TV di- 
rectly, which might imply that any correction would be uni- 
form, with little bias expected for estimated parameters. 

In practice this correction does not work. This is prob- 
ably because the exponential dependence of the correction 
on aN makes it extremely sensitive to the assumptions made 
in the derivation above. These assumptions are not correct. 
For example, it seems that the correlation between log(JV) 
and the associated error is important. 



Figure 2. A model including Malmquist bias. The bold, solid line 
is the original sample, the bold, dashed line is the distribution 
after processing as described in the text. Each curve is shown 
twice, but the upper right plot has both axes scaled by a factor 
of 3 and only shows data for 12.5 < log(A r ) < 16. Reference lines 
showing the evolution expected for j3 = 1.5 and 1.45 (dashed) are 
also shown (centre). 



2.2.3 An Estimation of the Malmquist Bias 

It is possible to do a simple numerical simulation to gauge 
the magnitude of the effect of Malmquist bias. A popula- 
tion of ten million column densities were selected at ran- 
dom from a power law distribution (/3 = 1.5, log(JV mm ) = 
10.9, log(Amax) = 22.5) as an 'unbiased' sample. Each line 
was given an 'observed' column density by adding a ran- 
dom error distributed with a normal or log-normal (for lines 
where 13.8 < log(jV) < 17.8) distribution, with a mean of 
zero and a standard deviation in log(TV) of 0.5. This proce- 
dure is a simple approximation to the type of errors coming 
from the curve of growth analysis discussed above, assuming 
that errors are approximately constant in log(A r ) (figure Q). 
The size of the error is larger than typical, so any inferred 
change in j3 should be an upper limit. 

Since a power-law distribution diverges as N — > a nor- 
mal distribution of errors in N would give an infinite number 
of observed lines at every column density. This is clearly un- 
physical (presumably the errors are not as extended as in 
a normal distribution and the population has some low col- 
umn density limit). Because of this the 'normal' errors above 
were actually constrained to lie within 3 standard deviations 
of zero. 

The results (figure ^) show that Malmquist bias has 
only a small effect, at least for the model used here. The 
main solid line is the original sample, the dashed line is 
the observed population. Note that the results in this paper 
come from fitting to a sample of lines with 12.5 < log(iV) < 
16 (section ^) — corresponding to the data shown expanded 
to the upper right of the figure. Lines with smaller column 
densities are not shown since that fraction of the population 
is affected by the lower density cut-off in the synthetic data. 

A comparison with the two reference lines, showing the 
slopes for a population with j3 = 1.5 or 1.45 (dashed), in- 
dicates that the expected change in /3 is ~ 0.05. The popu- 
lation of lines within the logarithmic region of the curve of 
growth appears to be slightly enhanced, but otherwise the 
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two curves are remarkably similar. The variations at large 
column densities are due to the small number of strong lines 
in the sample. 

2.2.4 Other Approaches 

What other approaches can be used to measure or correct 
the effects of Malmquist bias and line blending? Press & 
Rybicki (1993) used a completely different analysis of the 
Lyman-a forest. Generating and reducing synthetic data, 
with a known background and cloud population, would allow 
us to assess the effect of blending. Changing (sub-setting) 
the sample of lines that is analysed will alter the relative 
(and, possibly, absolute) importance of the two effects. 

The procedure used by Press & Rybicki (1993) is not 
affected by Malmquist bias or line blending, but it is difficult 
to adapt to measure the ionizing background. 

Profile fitting to high-resolution data is a slow pro- 
cess, involving significant manual intervention (we have tried 
to automate profile-fitting with little success). An accurate 
measurement of the systematic error in the ionizing back- 
ground would need an order of magnitude more data than 
is used here to get sufficiently low error limits. Even if this 
is possible — the analysis would need a prohibitive amount 
of CPU time — it would be sufficient work for a separate, 
major paper (we would be glad to supply our software to 
anyone willing to try this). 

Taking a sub-set of the data is not helpful unless it 
is less likely to be affected by the biases described above. 
One approach might be to reject points with large errors, or 
large relative errors, in column density since these are more 
affected by Malmquist bias. However, this would make the 
observations incomplete in a very poorly understood way. 
For example, relative errors are correlated with column den- 
sity (as noted above) and so rejecting lines with larger rela- 
tive errors would preferentially reject higher column density 
lines. There is no sense in trying to measure one bias if doing 
so introduces others. 

Unlike rejecting lines throughout the sample, changing 
the completeness limit does not alter the coverage of the ob- 
servations (or rather, it does so in a way that is understood 
and corrected for within the analysis) . Raising the complete- 
ness limits should make line blending less important since 
weaker lines, which are most likely to be blended, are ex- 
cluded from the fit. Whether it affects the Malmquist bias 
depends on the distribution of errors. 

For blended lines, which tend to be weak, raising the 
completeness limit should increase the absolute value of f3 
since the more populous region of the (hypothetical) power- 
law population of column densities will no longer be arti- 
ficially depleted. The effect on 7 is more difficult to assess 
since it is uncertain whether the completeness limits are cor- 
rect at each redshift. If the limits increase too rapidly with 
redshift, for example, then raising them further will reduce 
blending most at lower redshifts, lowering 7. But if they are 
increasing too slowly then the converse will be true. 



bias and line blending — will continue be a problem for any 
analysis of the Lyman-a forest. However, from the argu- 
ments above, it is likely that the effect of Malmquist bias is 
small and that, by increasing the completeness limit, we can 
assess the magnitude of the effect of line blending. 



2.3 Flux Calculations 

2.3.1 Galactic Extinction 

Extinction within our Galaxy reduces the apparent luminos- 
ity of the quasars and so lowers the estimate of the back- 
ground. Since the absorption varies with frequency this also 
alters the observed spectral slope. 

Observed fluxes were corrected using extinction esti- 
mates derived from the H I measurements of Heiles & Cleary 
(1979) for Q2204-573 and Stark et al. (1992) for all the other 
objects. H I column densities were converted to E(B — V) 
using the relationships: 



E(B -V) = 
E(B -V) = 



H I 



5.27 10 21 
4.37 10 21 



if 



H 1 



5.27 10 21 
otherwise 



< 0.1 



(9) 
(10) 



where the first value comes from Diplas & Savage (1994) 
and the second value, which compensates for the presence of 
H2, is the first scaled by the ratio of the conversions given in 
Bohlin, Savage & Drake (1978). A ratio R = A(V)/E{B-V) 
of 3.0 (Lockman & Savage 1995) was used and variations 
of extinction with frequency, A(X)/A(V) were taken from 
Cardelli, Clayton & Mathis (1989). 

The correction to the observed index, a , of the power- 
law continuum, 

/„ oc u~ a , (11) 
was calculated using 

A(V) d A(v) 

a ° = a + — m^A[v) (12) 

which, using the notation of Cardelli, Clayton & Mathis 
(1989), becomes 



MV) , ,„„ s d f . . b(x] 



2.3.2 Extinction in Damped Systems 



(13) 



Two quasars are known to have damped absorption systems 
along the line of sight (Wolfe et al. 1995). The extinction 
due to these systems is not certain, but model E includes 
the corrections listed in table These have been calculated 
using the SMC extinction curve in Pei (1992), with a cor- 
rection for the evolution of heavy element abundances taken 
from Pei & Fall (1995). The SMC extinction curve is most 
suitable for these systems since they do not appear to have 
structure at 2220 A (Boisse & Bergeron 1988), unlike LMC 
and Galactic curves. 



2.2.5 Conclusions 

Until either profile-fitting is automated, or the method of 
Press & Rybicki (1993) can be modified to include the prox- 
imity effect, these two sources of uncertainty — Malmquist 



2.3.3 Absorption by Clouds near the Quasar 

The amount of ionizing flux from the background quasar 
incident on a cloud is attenuated by all the other clouds 
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Object log(JV HI ) z ahs A(V) A Q 

Q0000-263 21.3 3.39 0.10 0.078 

Q2206-199 20.7 1.92 0.14 0.10 

20.4 2.08 0.019 0.049 

Table 1. The damped absorption systems and associated cor- 
rections (at 1450 A in the quasar's rest-frame) for model E. 



towards the source. If one of the intervening clouds has a 
large column density this can significantly reduce the extent 
of the effect of the quasar. 

To correct for this the fraction of ionizing photons from 
the quasar not attenuated by the intervening H I and He II 
absorption is estimated before fitting the model. A power- 
law spectrum is assumed and the attenuation is calculated 
for each cloud using the cross-sections given in Osterbrock 
(1989). The ratio n(He II)/n(H I) within the clouds will de- 
pend on several unknown factors (the true energy distribu- 
tion of the ionizing flux, cloud density, etc.), but was as- 
sumed to be 10 (Sargent et al. 1980, Miralda-Escude 1993). 

The attenuation is calculated using all the observed 
intervening clouds. This includes clouds which are not in- 
cluded in the main fit because they lie outside the column 
density limits, or are too close to the quasar (Az < 0.003). 

For most clouds (log(TV) ~ 13.5) near enough to the 
quasar to influence the calculation of the background this 
correction is unimportant (less than 1%). However, large 
(log(TV) ~ 18 or larger) clouds attenuate the flux to near 
zero. This explains why clouds with Af ~ 1 are apparent 
close to the QSO in figure [ll]. 

This relatively sudden change in optical depth at 
log(JV) ~ 18 is convenient since it makes the correc- 
tion insensitive to any uncertainties in the calculation (eg. 
n(He II)/n(H I), the shape of the incident spectrum, ab- 
sorption by heavier elements) — for most column densities 
any reasonable model is either insignificant (log (AT) < 17) 
or blocks practically all ionizing radiation (log(iV) > 19). 

In fact, the simple correction described above is in rea- 
sonable agreement with CLOUDY models, for even the most 
critical column densities. A model cloud with a column den- 
sity of log(iV) = 13.5 and constant density was irradiated 
by an ionizing spectrum based on that of Haardt & Madau 
(1996). Between the cloud and quasar the model included an 
additional absorber (constant density, log(iV) = 18) which 
modified the quasar's spectrum. The effect of the absorber 
(for a range of heavy element abundances from pure H to 
primordial to 0.1 solar) on the ionized fraction of H I was 
consistent with an inferred decrease in the quasar flux of 
about 80%. In comparison, the correction above, using a 
power-law spectrum with a — 1, gave a reduction of 60% 
in the quasar flux. These two values are in good agreement, 
considering the exponential dependence on column densities 
and the uncertainty in spectral shape. At higher and lower 
absorber column densities the agreement was even better, 
as expected. 



2.4 Redshift Corrections 

Gaskell (1982) first pointed out a discrepancy between the 
redshifts measured from Lyman a and C IV emission, and 
those from lower ionization lines (eg. Mg II, the Balmer se- 
ries). Lower ionization lines have a larger redshift. If the 



systemic redshift of the quasar is assumed to be that of the 
extended emission (Heckman et al. 1991), molecular emis- 
sion (Barvainis et al. 1994), or forbidden line emission (Car- 
swell et al. 1991), then the low ionization lines give a better 
measure of the rest-frame redshift. 

Using high ionization lines gives a reduced redshift for 
the quasar, implies a higher incident flux on the clouds from 
the quasar, and, for the same local depletion of lines, a higher 
estimate of the background. 

Espey (1993) re-analysed the data in Lu, Wolfe & Turn- 
shek (1991), correcting systematic errors in the quasar red- 
shifts. The analysis also considered corrections for optically 
thick and thin universes and the differences between the 
background and quasar spectra, but the dominant effect in 
reducing the estimate from 174 to 50 J23 was the change in 
the quasar redshifts. 

To derive a more accurate estimate of the systemic ve- 
locity of the quasars in our sample we made use of published 
redshift measurements of low ionization lines, or measured 
these where spectra were available to us. The lines used de- 
pended on the redshift and line strengths in the data, but 
typically were one or more of Mg II 2798 A, O 1 1304 A, and 
C II 1335 A. 

When no low ionization line observations were available 
(Q0420-388, Q1158-187, Q2204-573) we applied a mean 
correction to the high ionization line redshifts. These cor- 
rections are based on measurements of the relative velocity 
shifts between high and low ionization lines in a large sample 
of quasars (Espey & Junkkarinen 1996). They find a correla- 
tion between quasar luminosity and mean velocity difference 
(A„) with an empirical relationship given by: 



A v = exp(0.66 log L1450 — 13.72) kms 



(14) 



where L1450 is the rest-frame luminosity (ergs Hz 1 s 1 ) 
of the quasar at 1450 A for qo = 0.5 and Ho = 
100 kms _1 /Mpc. 



3 THE DATA 

Objects, redshifts and fluxes are listed in table |^. A total 
of 1675 lines from 11 quasar spectra were taken from the 
literature. Of these, 844 lie within the range of redshifts and 
column densities listed in table || although the full sample 
is used to correct for ab sorpti on between the quasar and 
individual clouds (section 2.3.3). The lower column density 



limits are taken from the references; upper column densities 
are fixed at log(iV) = 16 to avoid the double power law 
distribution discussed by Petitjean et al. (1993). Fluxes are 
calculated using standard formulae, assuming a power law 
spectrum (/„ oc v~ a ), with corrections for reddening. Low 
ionization line redshifts were used where possible, otherwise 
high ionization lines were corrected using the relation given 
in section 2.4. Values of a uncorrected for absorption are 
used where possible, corrected using the relation above. If 
no a was available, a value of 0.5 was assumed (Francis 
1993). 

References and notes on the calculations for each object 
follow: 

Q0000-263 Line list from Cooke (1994). There is some un- 
certainty in the wavelength calibration for these data, but 
the error (~ 30kms _1 ) is much less than the uncertainty 
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L„(1450) Typical change in log(J23) 
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42.8x10 s1 


13.3x10 s1 


-0.35 


+0.06 


+0.03 


-0.25 


Q2206-199 


2.574 


0.50 


19.4x10 s1 


6.3xl0 31 


-0.31 


+0.05 


+0.03 


-0.22 


Mean 


3.081 


0.56 


19.1x10 s1 


5.7xl0 31 


-0.24 


+0.12 


+0.07 


-0.04 



Table 2. The systemic redshifts, power law continuum exponents, and rest frame luminosities (ergs Hz -1 s — 1 at 1450 A) for the quasars 
used. Ho = 100 kms _1 /Mpc and luminosity scales as H^ 2 . The final four columns are an estimate of the relative effect of the various 
corrections in the paper (systemic redshift, correction for reddening to flux and spectral slope). 



in the quasar redshift (~ 900km s 1 ) w hich is taken into 



account in the error estimate (section 4.2). Redshift this pa- 
per (section 2.4). Flux and a measurements from Sargent, 
Steidel & Boksenberg (1989). 

Q0014+813 Line list from Rauch et al. (1993). Redshift 
this paper (section 2.4). Flux and a measurements from Sar- 
gent, Steidel & Boksenberg (1989). 

Q0207-398 Line list from Webb (1987). Redshift (O I line) 
from Wilkes (1984). Flux and a measurements from Baldwin 
et al. (1995). 

Q0420-388 Line list from Atwood, Baldwin & Carswell 
(1985). Redshift, flux and a from Osmer (1979) (flux mea- 
sured from plot). The redshifts quoted in the literature vary 



significantly, so a larger error (0.01) was used in section 4.2 
Q 1033-033 Line list and flux from Williger et al. (1994) 
From their data, a = 0.78, with out a reddening correction 
Redshift this paper (section 2.4). 



Q 1100-264 Line list from Cooke (1994). Redshift from Es- 
pey et al. (1989) and a from Tytler & Fan (1992). Flux 
measured from Osmer & Smith (1977). 

Q1158-187 Line list from Webb (1987). Redshift from 
Kunth, Sargent & Kowal (1981). Flux from Adam (1985). 

Q 1448-232 Line list from Webb (1987). Redshift from Es- 
pey et al. (1989). Flux and a measured from Wilkes et al. 
(1983), although a wide range of values exist in the litera- 
ture and so a larger error (0.6 magnitudes in the flux) was 
used in section L2. 

Q2000-330 Line list from Carswell et al. (1987). Redshift 



this paper (section 2.4). Flux and a measurements from Sar- 
gent, Steidel & Boksenberg (1989). 

Q2204-573 Line list from Webb (1987). Redshift from 
Wilkes et al. (1983). V magnitude from Adam (1985). 

Q2206-199 Line list from Rauch et al. (1993). Redshift this 
paper (section 2.4). V magnitude from Hewitt & Burbidge 
(1989). 

Table |^ also gives an estimate of the relative effect of the 
different corrections made here. Each row gives the typical 
change in log(J23) that would be estimated using that quasar 
alone, with a typical absorption cloud 2 Mpc from the quasar 
(qo = 0.5, Ho = 100 kms -1 ). The correction to obtain the 
systemic redshift is not necessary for any quasar whose red- 
shift has been determined using low ionization lines. In such 



Object 




N 




z 


Number 


name 


Low 


High 


Low 


High 


of lines 


Q0000-263 


14.00 


16.00 


3.1130 


3.3104 


62 








3.4914 


4.1210 


101 


Q0014+813 


13.30 


16.00 


2.7000 


3.3800 


191 


Q0207-398 


13.75 


16.00 


2.0765 


2.1752 


11 








2.4055 


2.4878 


7 








2.6441 


2.7346 


6 








2.6852 


2.7757 


9 








2.7346 


2.8180 


8 


Q0420-388 


13.75 


16.00 


2.7200 


3.0800 


73 


Q1033-033 


14.00 


16.00 


3.7000 


3.7710 


16 








3.7916 


3.8944 


21 








3.9191 


4.0301 


24 








4.0548 


4.1412 


25 








4.1988 


4.3139 


30 








4.3525 


4.4490 


23 








4.4517 


4.4780 


2 


Ql 100-264 


12.85 


16.00 


1.7886 


1.8281 


2 








1.8330 


1.8733 


8 








1.8774 


1.9194 


13 








1.9235 


1.9646 


9 








1.9696 


2.0123 


10 








2.0189 


2.0617 


6 








2.0683 


2.1119 


18 


Q1158-187 


13.75 


16.00 


2.3397 


2.4510 


9 


Q1448-232 


13.75 


16.00 


2.0847 


2.1752 


9 


Q2000-330 


13.75 


16.00 


3.3000 


3.4255 


23 








3.4580 


3.5390 


15 








3.5690 


3.6440 


18 








3.6810 


3.7450 


11 


Q2204-573 


13.75 


16.00 


2.4467 


2.5371 


10 








2.5454 


2.6276 


12 








2.6441 


2.7280 


8 


Q2206-199 


13.30 


16.00 


2.0864 


2.1094 


2 








2.1226 


2.1637 


8 








2.1760 


2.2188 


5 








2.2320 


2.2739 


7 








2.2887 


2.3331 


7 








2.3471 


2.3940 


10 








2.4105 


2.4574 


4 








2.4754 


2.5215 


11 


Total: 11 quasars 








844 



Table 3. Completeness limits. 
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Without Inv. Eff. With Inv. Eff. 

Variable Statistic Prob. Statistic Prob. 

N 1.11 0.17 1.05 0.22 

z 1.12 0.16 1.05 0.22 

Table 4. The K-S statistics measuring the quality of the fit. 



by the majority of lines which are not influenced b y the 
quasar. The likelihood ratio test that we use in section |6.2.1 



is a more powerful method for comparing two models, but 
can only be used if the models are already a reasonable fit 
(as shown here). 



cases the value given is the expected change if the redshift 
measurement had not been available. 

Using the systematic redshift always reduces the back- 
ground estimate, while correcting for reddening always acts 
in the opposite sense. The net result, in the final column 
of table H, depends on the relative strength of these two 
factors. For most objects the redshift correction dominates, 
lowering log(J23) by ~ 0.15 (a decrease of 30%), but for four 
objects the reddening is more important (Q0014+813, the 
most reddened, has B — V = 0.33; the average for all other 
objects is 0.09). 

Figure ^ shows the distribution of column density, N, 
and redshift, z, for the lines in the sample. The completeness 
limit was taken from the literature and depends on the qual- 
ity of the spectra. There is also a clear trend with redshift 
as the number density increases and weak lines become less 
and less easy to separate in complex blends, whatever the 
data quality (see section 2.1 for a more detailed discussion 
of line blending) . 



4 FIT QUALITY AND ERROR ESTIMATES 
4.1 The Quality of the Fit 

Figures | and | show the cumulative data and model for 
each variable using two models: one includes the proximity 
effect (model B), one does not (model A). The probabilities 
of the associated K-S statistics are given in table ^[ For the 
column density plots the worst discrepancy between model 
and data occurs at log(iV) = 14.79. The model with the 
proximity effect (to the right) has slightly more high col- 
umn density clouds, as would be expected, although this is 
difficult to see in the figures (note that the dashed line — 
the model — is the curve that has changed) . In the redshift 
plots the difference between the two models is more appar- 
ent because the changes are confined to a few redshifts, near 
the quasars, rather than, as in the previous figures, spread 
across a wide range of column densities. The apparent dif- 
ference between model and data is larger for the model that 
includes the proximity effect (on the right of figure ||). How- 
ever, this is an optical illusion as the eye tends to measure 
the vertical difference between horizontal, rather than diag- 
onal, lines. In fact the largest discrepancy in the left figure is 
at z = 3.323, shifting to z = 3.330 when the proximity effect 
is included. It is difficult to assess the importance of indi- 
vidual objects in cumulative plots, but the main difference 
in the redshift figure occurs near the redshift of Q0014+813. 
However, since this is also the case without the proximity ef- 
fect (the left-hand figure) it does not seem to be connected 
to the unusually large flux correction for this object (sec- 
tion §). 

In both cases — with and without the proximity ef- 
fect — the model fits the data reasonably well. It is not 
surprising that including the proximity effect only increases 
the acceptability of the fit slightly, as the test is dominated 



4.2 Sources of Error 

There are two sources of stochastic uncertainty in the values 
of estimated parameters: the finite number of observations 
and the error associated with each observation (column den- 
sities, redshifts, quasar fluxes, etc.). 

The first source of variation — the limited information 
available from a finite number of observations — can be as- 
sessed by examining the distribution of the posterior proba- 
bilities for each parameter. This is described in the following 
section. 

The second source of variation — the errors associated 
with each measurement — can be assessed by repeating the 
analysis for simulated sets of data. In theory these errors 
could have been included in the model and their contribution 
would have been apparent in the posterior distribution. In 
practice there was insufficient information or computer time 
to make a detailed model of the error distribution. 

Instead, ten different sets of line-lists were created. 
Each was based on the original, with each new value, X, 
calculated from the observed value x and error estimate ax'- 



X — x + aax 



(15) 



where a was selected at random from a (approximate) nor- 
mal distribution with zero mean and unit variance. The red- 
shift (standard error 0.003) and luminosity (standard er- 
ror 0.2 magnitudes) of each background quasar were also 
changed. For Q0420-388 the redshift error was increased to 
0.1 and, for Q1448-232, the magnitude error was increased 
to 0.6 magnitudes. The model was fitted to each data set 
and the most likely values of the parameters recorded. A 
Gaussian was fitted to the distribution of values. In some 
cases (eg. figure ^) a Gaussian curve may not be the best 
way to describe the distribution of measurements. However, 
since the error in the parameters is dominated by the small 
number of data points, rather than the observational errors, 
using a different curve will make little difference to the final 
results. 

Since the two sources of stochastic error are not ex- 
pected to be correlated they can be combined to give the 
final distribution of the parameters. The Gaussian fitted to 
the variation from measurement errors is convolved with the 
posterior distribution of the variable. The final, normalized 
distribution is then a good approximation to the actual dis- 
tribution of values expected. 

This procedure is shown in figures to ^. For each pa- 
rameter in the model the 'raw' posterior distribution is plot- 
ted (thin line and points). The distribution of values from 
the synthetic data is shown as a dashed histogram and the 
fitted Gaussian is a thin line. The final distribution, after 
convolution, is the heavy line. In general the uncertainties 
due to a finite data set are the main source of error. 
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Figure 3. The lines in the full sample (left) used to calculate the attenuation of the quasar flux by intervening clouds and the restricted 
sample (right) to which the model was fitted. 




13.0 13.5 14.0 14.5 15.0 15.5 13.0 13.5 14.0 14.5 15.0 15.5 

log Column Density log Column Density 



Figure 4. The cumulative data (solid line) and model (dashed), integrating over z, for the lines in the sample, plotted against column 
density (log(iV)). The model on the right includes the proximity effect. 




Redshift Redshift 



Figure 5. The cumulative data (solid line) and model (dashed), integrating over N, for the lines in the sample, plotted against redshift. 
The model on the right includes the proximity effect. 
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4.3 Error Estimates from Posterior Probabilities 

If p(y\0) is the likelihood of the observations (y), given the 
model (with parameters 9), then we need an expression for 
the posterior probability of a 'parameter of interest', r\. This 
might be one of the model parameters, or some function of 
the parameters (such as the background flux at a certain 
redshift): 



V = 9(0) 



(16) 



For example, the valu e of J23 at a particular redshift for 
models C to E in section 2.1 is a linear function of several 
parameters (two or more of J3.25 , Jz B , Qi , and a.2). To calcu- 
late how likely a particular flux is the probabilities of all the 
possible combinations of parameter values consistent with 
that value must be considered: it is necessary to integrate 
over all possible values of f3 and 7', and all values of J ZB , oti, 
etc. which are consistent with J23(z) having that value. 

In other words, to find the posterior distribution of r], 
n(r]\6), we must marginalise the remaining model parame- 
ters: 

7r(#) = lim - [ iv(9\y)d8 , (17) 
1 Jd 

where D is the region of parameter space for which r\ < 
gifi) < 77 + 7 and 7r(#|y) oc n{9)p(y\9), the posterior density 
of 9 with prior n(9). 

A uniform prior is used here for all parameters (equiv- 
alent to normal maximum likelihood analysis). Explicitly, 
power law exponents and the logarithm of the flux have 
prior distributions which are uniform over [—00, +00]. 

Doing the multi-dimensional integral described above 
would require a large (prohibitive) amount of computer 
time. However, the log-likelihood can be approximated by 
a second order series expansion in 9. This is equivalent to 
assuming that the other parameters are distributed as a mul- 
tivariate normal distribution, and the result can then be cal- 
culated analytically. Such a procedure is shown, by Leonard, 
Hsu & Tsui (1989), to give the following procedure when 
g(9) is a linear function of 6: 



7r(??|y) » 

where 
7TM(f?|y) 



7TM( 



|R v | 1 /2(6 I/ T R„- 1 6„)i/2 



sup 7r(0|y) 

8:g(fl)=l7 



K = 



T(^|y) 

dg(6) 



d 2 hl7T(%) 



d{99' 1 



(18) 

(19) 
(20) 
(21) 

(22) 



The likelihood is maximised with the constraint that g(9) 
has a particular value. R„ is the Hessian matrix used in the 
fitting routine (Press et al. 1992) and b v is known (when r\ is 
the average of the first two of three parameters, for example, 
b v = 0.5,0.5,0). 

This quickens the calculation enormously. To estimate 
the posterior distribution for, say, J23, it is only necessary 
to choose a series of values and, at each point, find the best 
fit consistent with that value. The Hessian matrix, which is 



returned by many fitting routines, can then be used — fol- 
lowing the formulae above — to calculate an approximation 
to the integral, giving a value proportional to the probabil- 
ity at that point. Once this has been repeated for a range of 
different values of J23 the resulting probability distribution 
can be normalised to give an integral of one. 

Note that this procedure is only suitable when g(9) is a 
linear function of 9 — Leonard, Hsu & Tsui (1989) give the 
expressions needed for more complex parameters. 



5 RESULTS 

A summary of the results for the different models is given 
in table ^. The models are: 

A — No Proximity Effect. The population model de- 
scribed in section |j| but without the proximity effect. 

B — Constant Background. The population model de- 
scribed in section |s| with a constant ionising background. 

C — Power Law Background. The population model de- 
scribed in section]^ with an ionising background which varies 
as a power law with redshift 

D — Broken Power Law Background. The population 
model described in section [| with an ionising background 
whose power law exponent changes at zb = 3.25. 

E — Correction for Extinction in Damped Systems. As 
D, but with a correction fo r abs orption in known damped 
absorption systems (section 2.3.2). 



0.5 and H t 







In this paper we assume go 
100 kms _1 /Mpc. 

5.1 Population Distribution 

The maximum likelihood 'best-fit' values for the parameters 
are given in table ^[ The quoted errors are the differences 
(a single value if the distribution is symmetric) at which the 
probability falls by the factor 1/Ve. This is equivalent to a 
'lcr error' for parameters with normal error distributions. 

The observed evolution of the number of clouds per unit 
redshift is described in the standard notation found in the 
literature 



dN/dz = A (l + z)"' 



(23) 



The variable used in the maximum likelihood fits here, 7', 
excludes variations expected from purely cosmological vari- 
ations and is related to 7 by: 



7 : 



7' + 1 if go = 



7' + 



if g = 0.5 



(24) 



Figure |9j shows the variation in population parameters 
for model D as the completeness limits are increased in steps 
of Alog(iV) = 0.1. The number of clouds decreases from 844 
to 425 (when the completeness levels have been increased by 
Alog(iV) = 0.5). 



5.2 Ionising Background 

Values of the ionising flux parameters are show in table ^[ 
The expected probability distributions for models B and D 
are shown in figures |^ and |^. The background flux relation 
is described in section |2.l| . 
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Figure 7. The expected probability distribution of the log background flux at z = 3.25 (heavy line) for models B (left) and D. The 
uncertainty from the small number of lines near the qua sar (line with points) is significantly larger than that from uncertainties in column 
densities or quasar properties (thin curve). See section for a full description of the plot. 
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Model 




(8 




7 










-2 log-likelihood 


A 


1.66 


±0.03 


2.7 


±0.3 






No background 




60086.2 


B 


1.67 


±0.03 


2.9 


±0.3 


-21.0 


±0.2 






60052.8 


C 


1.67 


±0.03 


3.0 


±0.3 


-21.0 


±0.2 


-1 ±3 




60052.6 


D 


1.67 


±0.04 


3.0 


±0.3 


-20.9 


±0.3 


+5,-6 -2 ±4 


3.25 


60052.4 


E 


1.67 


±0.03 


3.0 


±0.3 


-20.9 


+0.3,-0.2 


+5,-6 -2 +7,-4 


3.25 


60051.4 



Table 5. The best— fit parameters and expected errors for the models. 




1.60 1.70 1.80 1.90 2.00 1.5 2.0 2.5 3.0 3.5 4.0 

P y 



Figure 9. The expected probability distribution of the population parameters for model D. The top curve is for all data, each lower 
curve is for data remaining when the column density completeness limits are progressively increased by Alog(iV) =0.1. 



The variables used to describe the variation of the flux 
with redshift are strongly correlated. To illustrate the con- 
straints mor e clearly the marginalised posterior distribution 
(section of J23 was calculated at a series of redshifts. 
These are shown (after convolution with the combination of 
Gaussians appropriate for the uncertainties in the param- 
eters from observational errors) for model D in figure |Io[ 
The distribution at each redshift is calculated independently. 
This gives a conservative representation since the marginal- 
isation procedure assumes that parameters can take all pos- 
sible values consistent with the background at that redshift 
(the probability that the flux can be low at a certain red- 
shift, for example, includes the possibility that it is higher 
at other redshifts). Figure [j^ also compares the results from 
the full data set (solid lines and smaller boxes) with those 
from the data set with column density completeness limits 
raised by A log(iV) = 0.5 (the same data as the final curves 
in figure |J) . 

Table ^ gives the most likely flux (at probability p m ), 
an estimate of the 'ler error' (where the probability drops to 
Praj \fe), the median flux, the upper and lower quartiles, and 
the 5% and 95% limits for model D at the redshifts shown 
in figure [To| It is difficult to assess the uncertainty in these 
values. In general the central measurements are more reli- 
able than the extreme limits. The latter are more uncertain 
for two reasons. First, the distribution of unlikely m odel s is 
more likely to be affected by assumptions in section 4.3 on 
the normal distribution of secondary parameters. Second, 
the tails of the probability distribution are very flat, making 
the flux value sensitive to numerical noise. Extreme limits, 
therefore, should only be taken as a measure of the relevant 
flux magnitude. Most likely and median values are given to 





2 = 2 


2 = 2.5 


2 = 3 2 = 


3.5 


2 = 4 


2 = 4.5 


Pm/Ve 


30 


50 
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60 
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30 


Pm 
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80 


63 
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5% 
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50 


40 


30 


20 
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70 


80 
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10 
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87 


75 
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1000 


400 
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160 


100 


200 


95% 


30000 


3000 


400 


300 


400 


600 


Table 6. 


The fluxes (J23) corresponding 


to 


various 


aosterior 



probabilities for model D. See the text for details on the expected 
errors in these values. 



the nearest integer to help others plot our results — the 
actual accuracy is probably lower. 



6 DISCUSSION 

6.1 Population Parameters 

Parameter values for the different models are given in table 
|E| They are generally consistent with other estimates (Lu, 
Wolfe & Turnshek 1991; Rauch et al. 1993). Including the 
proximity effect increases 7 by ~ 0.2. Although not statis- 
tically significant, the change is in the sense expected, since 
local depletions at the higher redshift end of each data set 
are removed. 

Figure ^| shows the change in population parameters as 
the completeness limits for the observations are increased. 
The most likely values (curve peaks) of both /3 and 7 increase 
as weaker lines are excluded, although 7 decreases again for 
the last sample. 

The value of f3 found here (~ 1.7) is significantly differ- 
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Figure 10. The expected probability distribution of the log background flux for model D, comparing the results from the full data set 
with those obtained when the column density completeness limit is raised by A log(iV) = 0.5 (dashed line, left; larger boxes, right). The 
box plots show median, quartiles, and 95% limits. 



ent from that found by Press & Rybicki (1993) (J3 ~ 1.4) us- 
ing a different technique (which is insensitive to Malmquist 
bias and line blending). The value of j3 moves still fur- 
ther away as the column density completeness limits are in- 
creased. This is not consistent with Malmquist bias, which 
would give a smaller change in j3 (section ^^), but could 
be a result of either line blending or a population in which 
(3 increases with column density. The latter explanation is 
also consistent with Cristiani et al. (1995) who found a 
break in the column density distribution with j3 = 1.10 for 
log(7V)< 14.0, and j3 = 1.80 abo ve thi s value. Later work 
(Giallongo et al. 1996, see section 6.3.1) confirmed this. 

Recent work by Hu et al. (1995), however, using data 
with better signal-to-noise and resolution, finds that the 
distribution of column densities is described by a single 
power law (/3 ~ 1.46) until log(JV) ~ 12.3, when line blend- 
ing in in their sample becomes significant. It might be pos- 
sible that their sample is not sufficiently large (66 lines with 
log(JV) > 14.5, compared with 192 here) to detect a steeper 
distribution of high column density lines. 

The change in 7 as completeness limits are raised may 
reflect the decrease in line blending at higher column densi- 
ties. This suggests that the value here is an over-estimate, 
although the shift is within the 95% confidence interval. No 
estimate is significantly different from the value of 2.46 found 
by Press & Rybicki (1993) (again, using a method less sus- 
ceptible to blending problems) . 



6.2 The Proximity Effect 

6.2.1 Is the Proximity Effect Real? 

The likelihood ratio statistic (equivalent to the 'F test'), 
comparing model A with any other, indicates that the null 
hypothesis (that the proximity effect, as described by the 
model here, should be disregarded) can be rejected with a 
confidence exceeding 99.9%. Note that this confirmation is 
based on the likelihood values in table ^. This test is much 
more powerful than the K-S test (section [f.l| ) which was 
only used to see whether the models were sufficiently good 
for the likelihood ratio test to be used. 



To reiterate: if model A and model B are taken as com- 
peting descriptions of the population of Lyman-a clouds, 
then the likelihood ratio test, which allows for the extra de- 
gree of freedom introduced, strongly favours a description 
which includes the proximity effect. The model without the 
proximity effect is firmly rejected. This does not imply that 
the interpretation of the effect (ie. additional ionization by 
background radiation) is correct, but it does indicate that 
the proximity effect, in the restricted, statistical sense above, 
is 'real' (cf. Roser 1995). 

If the assumptions behind this analysis are correct, in 
particular that the proximity effect is due to the additional 
ionising flux from the quasar, then the average value of the 
background is 100 J23 (model B). 

If a more flexible model for the background (two 
power laws) is used the flux is consistent with a value of 



120 " 



J23 (model D at z = 3.25). 



6.2.2 Systematic Errors 

Five sources of systematic error are discussed here: 
Malmquist bias and line blending; reddening by damped 
absorption systems; increased clustering of clouds near 
quasars; the effect of gravitational lensing. 

The constraints on the background given here may be 
affe cted by Malmquist bias and line blending (sections 
and 



2.2 



6.1). The e ffects of line blending will be discussed fur- 
ther in section |6.3.2 , where a comparison with a different 



procedure suggests that it may cause us to over-estimate 
the flux (by perhaps 0.1 dex). Malmquist bias is more likely 
to affect parameters sensitive to absolute column densities 
than those which rely only on relative changes in the ob- 
served population. So while this may have an effect on /3, it 
should have much less influence on the inferred background 
value. 

Attenuation by intervening damped absorption systems 
will lower the apparent quasar flux and so give an estimate 
for the background which is too low. This is corrected in 
model E, which i nclud es adjustments for the known damped 
systems (section 2.3.2 , table [l]). The change in the inferred 
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background flux is insignificant (figure [t2| table ^]), implying 
that the magnitude of the bias is less than 0.1 dex. 

If quasars lie in regions of increased absorption line 
clustering (Loeb & Eisenstein 1995; Yurchenko, Lanzetta & 
Webb 1995) then the background flux may be overestimated 
by up to 0.5, or even 1, dex. 

Gravitational lensing may change the apparent bright- 
ness of a quasar — in general the change can make the 
quasar appear either brighter or fainter. Absorption line ob- 
servations are made towards the brightest quasars known 
(to get good quality spectra). Since there are more faint 
quasars than bright ones this will preferentially select ob- 
jects which have been brightened by l ensing (see the com- 
ments on Malmquist bias in section 2/2 ) . An artificially high 
estimate of the quasar flux will cause us to over-estimate 
the background. 

Unfortunately, models which assess the magnitude of 
the increase in quasar brightness are very sensitive to the 
model population of lensing objects. From Pei (1995) an 
upper limit consistent with observations is an increase in 
flux of about 0.5 magnitudes, corresponding to a background 
estimate which is too high by a factor of 1.6 (0.2 dex). The 
probable effect, however, could be much smaller (Blandford 
& Narayan 1992). 

If bright quasars are more likely to be lensed we can 
make a rudimentary measurement of the effect by splitting 
the data into two separate samples. When fitted with a con- 
stant background (model B) the result for the five brightest 
objects is indeed brighter than that for the remaining six, 
by 0.1 dex. The errors, however, are larger (0.3 dex), making 
it impossible to draw any useful conclusions. 

The effects of Malmquist bias, line blending and 
damped absorption systems are unlikely to change the re- 
sults here significantly. Cloud clustering and gravitational 
lensing could be more important — in each case the back- 
ground would be over-estimated. The magnitude of these 
last two biases is not certain, but cloud clustering seems 
more likely to be significant. 



6.2.3 Is there any Evidence for Evolution? 

More complex models allow the background flux to vary with 
redshift. If the flux does evolve then these models should fit 
the data better. However, there is no significant change in 
the fit when comparing the likelihood of models C to E with 
that of B. Nor are qi or 0:2 significantly different from zero. 
So there is no significant evidence for a background which 
changes with redshift. 

The asymmetries in the wings of the posterior distribu- 
tions of Qfi or «2 for model D (figure ^) are a result of the 
weak constraints on upper limits (see next section). The box 
plots in figure [to] illustrate the range of evolutions that are 
possible. 



6.2.4 Upper and Lower Limits 

While there is little evidence here for evolution of the back- 
ground, the upper limits to the background flux diverge 
more strongly than the lower limits at the lowest and highest 
redshifts. Also, the posterior probability of the background 
is extended more towards higher values. 



The background was measured by comparing its effect 
with that of the quasar. If the background were larger the 
quasar would have less effect and the clouds with Ap < 1 
would not need as large a correction to the observed column 
density for them to agree with the population as a whole. If 
the background was less strong then the quasars would have 
a stronger influence and more clouds would be affected. 

The upper limit to the flux depends on clouds influ- 
enced by the quasar. Figure [H] shows how A f changes with 
redshift and proximity to the background quasar. From this 
figure it is clear that the upper limit is dominated by only a 
few clouds. However, the lower limit also depends on clouds 
near to, but not influenced by, the quasar. This involves 
many more clouds. The lower limit is therefore stronger, 
more uniform, and less sensitive to the amount of data, than 
the upper limit. 

Other procedures for calculating the errors in the flux 
have assumed that the error is symmetrical (the only appar- 
ent exception is Fernandez-Soto et al. (1995) who unfortu- 
nately had insufficient data to normalize the distribution). 
While this is acceptable for B and 7, whose posterior prob- 
ability distributions (figure ph can be well-approximated by 
Gaussian curves, it is clearly wrong for the background (eg. 
figure |ic|) , especially where there are less data (at the lowest 
and highest redshifts). 

An estimate based on the assumption that the error is 
normally distributed will be biased in two ways. First, since 
the extended upper bound to the background has been ig- 
nored, it will underestimate the 'best' value. Second, since 
the error bounds are calculated from the curvature of the 
posterior distribution at its peak (ie. from the Hessian ma- 
trix) they will not take account of the extended 'tails' and 
so will underestimate the true range of values. In addition, 
most earlier work has calculated errors assuming that the 
other population parameters are fixed at their best-fit val- 
ues. This will also under-estimate the true error limits. All 
these biases become more significant as the amount of data 
decreases. 

The first of these biases also makes the interpretation 
of the box-plots (eg. figures [jl] and [j^) more difficult. For 
example, the curves in the left-hand plot in figure |l(] and the 
data in table ^ show that the value of the flux with highest 
probability at z — 2 is 140 J23 (for model D). In contrast 
the box-plot on the right shows that the median probability 
is almost twice as large (230 J23). Neither plot is 'wrong': 
this is the consequence of asymmetric error distributions. 



6.3 Comparison with Previous Estimates 

6. 3. 1 Earlier High-Resolution Work 

Fernandez-Soto et al. (1995) fitted high signal-to-noise data 
towards three quasars. For 2 < z < 2.7 they estimate an 
ionizing background intensity of 32 J23, with an absolute 
lower limit (95% confidence) of 16 J23 (figure |l^, the leftmost 
cross). They were unable to put any upper limit on their 
results. 

Cristiani et al. (1995) determined a value of 50 J23 using 
a sample of five quasars with a lower column density cut- 
off of log(iV) = 13.3. This sample was recently extended 
(Giallongo 1995). They find that the ionizing background 
is roughly constant over the range 1.7 < z < 4.0 with a 
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Figure 11. On the left, absorber redshifts are plotted against Ap. On the right Ap is plotted against the distance between cloud and 
quasar. Note that the correction for the quasar's flux, and hence the upper limit to the estimate of the background, is significant for only 
a small fraction of the clouds. 
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Figure 12. The expected probability distribution of the log background flux for models D (left) and E (right, including a correction 
for the known damped absorption systems). The box plots show median, quartiles, and 95% limits. The shaded area covers the range of 
backgrounds described in Fall & Pei (1995). The lower boundary is the expected background if all quasars are visible, the higher fluxes 
are possible if an increasing fraction of the quasar population is obscured at higher redshifts. The crosses and arrows mark the extent of 
previous measurements from high resolution spectra — see the text for more details. 



value of 50 J23 which they considered a possible lower limit 
(figure [l^, the middle lower limit). 

While this paper was being refereed Giallongo et al. 
(1996) became available, extending the work above. Using a 
maximum likelihood analysis with an unspecified procedure 
for calculating errors they give an estimate for the back- 



ground of 50 ± 10 J23 • They found no evidence for evolution 
with redshift when using a single power law exponent. 

Williger et al. (1994) used a single object, Q 1033-033, 
which is included in this sample, to give an estimate of 10 — 
30 J23 (figure [jj, the rightmost cross). The error limits are 
smaller than those found here, even though they only use 
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a subset of this data, which suggests that they have been 
significantly underestimated. 

If the errors in Williger et al. (1994) are indeed under- 
estimates then these measurements are consistent with the 
results here. However, the best-fit values are all lower than 
those found here. This may be, at least partly, because of 



the biases discussed in section 6.2.4. 

Williger et al. (1994) used a more direct method than 
usual to estimate the background. This gives a useful con- 
straint on the effect of line blending in the procedures used, 
and is explored in more detail below. 

6.3.2 Q 1033-033 and Line Blending 

The measured value of the background, 80 J23 (model 
D at z — 4) , is larger than an earlier estimate using a subset 
of this data (Williger et al. 1994, Q1033-033, 10 - 30 J 23 ). 

As has already been argued, it is difficult to understand 
how a procedure using much less data could have smaller 
error limits than the results here, so it is likely that the 
error was an underestimate and that the two results are 
consistent. However, it is interesting to see if there is also a 
systematic bias in the analyses used. 

The correction for galactic absorption is not very large 
for this object (about 20%). More importantly, the pro- 
cedures used differ significantly in how they are affected 
by blended lines. These are a problem at the highest red- 
shifts, where the increased Lyman-a cloud population den- 
sity means that it is not always possible to resolve indi- 
vidual clouds. Williger et al. (1994) added additional lines 
(log(iV) = 13.7, b = 30 kms" 1 ) to their z = 4.26 spectra of 
Q 1033-033 and found that between 40% and 75% would be 
missed in the line list. 

As the lower column density limit is raised Williger et al. 
(1994) find that the observed value of 7 also increases. The 
resulting stronger redshift evolution would make the deficit 
of clouds near the quasar more significant and so give a lower 
estimate of the background. Although not significant at the 
95% level, there is an indication that 7 also incr ease s with 
higher column density in this analysis (section 6.1, figure 
^|). While it is possible that 7 varies with column density 
the same dependence would be expected if line blending is 
reducing the number of smaller clouds. To understand how 
line blending can affect the estimates, we will now examine 
the two analyses in more detail. 

Line blending makes the detection of lines less likely. 
Near the quasar lines are easier to detect because the forest 
is more sparse. In the analysis used in this paper the ap- 
pearance of these 'extra' lines reduces the apparent effect of 
the quasar. Alternatively, one can say that away from the 
quasar line blending lowers 7. Both arguments describe the 
same process and imply that the estimated background flux 
is too large. 

In contrast, Williger et al. (1994) take a line-list from 
a crowded region, which has too few weak lines and cor- 
respondingly more saturated lines, and reduce the column 
densities until they agree with a region closer to the quasar. 
Since a few saturated lines are less sensitive to the quasar's 
flux than a larger number of weaker lines, the effect of this 
flux is over-estimated (and poorly determined) , making the 
background seem less significant and giving a final value 
for the background flux which is too small. This method is 



therefore biased in the opposite sense to ours and so the true 
value of the background probably lies between their estimate 
and ours. 

The comparison with Williger et al. (1994) gives one es- 
timate of the bias from line blending. Another can be mad e 
by raising the completeness limits of the data (section 2.2). 
This should decrease the number of weak, blended lines, 
but also excludes approximately half the data. In figure |l(j 
the flux estimates from the full data set are shown together 
with those from one in which the limits have been raised by 
A log(iV) = 0.5. There is little change in the lowest reason- 
able flux, an increase in the upper limits, and in increase 
in the 'best-fit' values. The flux for z < 3 is al most uncon- 
strained by the restricted sample (section 6.2.4 explains the 
asymmetry) . 

An increase of 0.5 in log(JV) is a substantial change in 
the completeness limits. That the lower limits remain con- 
stant (to within ~ 0.1 dex) suggests that line blending is 
not causing the flux to be significantly over-estimated. The 
increase in the upper limits is expected when the number of 
clouds in the sample decreases (section |6.2.4 ). 

In summary, the total difference between our measure- 
ment and that in Williger et al. (1994) is 0.7 dex which can 
be taken as an upper limit on the effect of line blending. 
However, a more typical value, from the constancy of the 
lower limits when completeness limits are raised, is proba- 
bly ~ 0.1 dex. 



6.3.3 Results from Lower Resolution Spectra 

Bechtold (1994) analysed lower resolution spectra towards 
34 quasars using equivalent widths rather than individual 
column density measurements. She derived a background 
flux of 300 J23 (1.6 < z < 4.1), decreasing to 100 J23 when a 
uniform correction was applied to correct for non-systemic 
quasar redshifts. With low-resolution data a value of (3 is 
used to change from a distribution of equivalent widths to 
column densities. If /? is decreased from 1.7 to a value closer 
to that found for narrower lines (see section 3.1) then the 
inferred background estimate could decrease further. 

The evolution was not well-constrained (—7 < a < 4). 
No distinction was made between the lower and upper con- 
straints on the flux estimate, and it is likely that the wide 
range of values reflects the lack of strong upper constraints 
which we see in our analysis. 

It is not clear to what extent this analysis is affected by 
line blending. Certainly the comments above — that rela- 
tively more clouds will be detected near the quasar — also 
apply. 



6.3.4 Lower Redshift Measurements 

The background intensity presented in this paper is much 
larger than the 8 J23 upper limit at z = found by Vogel 
et al. (1995). Kulkarni & Fall (1993) obtain an even lower 
value of 0.6 J23 at z = 0.5 by analysing the proximity 
effect in HST observations. However, even an unevolving 
flux will decrease by a factor of ~ 50 between 2 = 2 and 0, 
so such a decline is not inconsistent with the results given 
here. 
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6.4 What is the Source of the Background? 

6.4-1 Quasars 

Quasars are likely to provide a significant, if not dominant, 
contribution to the extragalactic background. An estimate 
of the ionizing background can be calculated from models of 
the quasar population. Figure [l^ shows the constraints from 
models D and E and compares them with the expected evo- 
lution of the background calculated by Fall & Pei (1995). 
The background can take a range of values (the shaded 
region), with the lower boundary indicating the expected 
trend for a dust-free universe and larger values taking into 
account those quasars that may be hidden from our view, 
but which still contribute to the intergalactic ionizing flux. 
The hypothesis that the flux is only from visible quasars (the 
unobscured model in Fall & Pei 1995) is formally rejected at 
over the 95% significance level since the predicted evolution 
is outside the 95% bar in the box plots at higher redshift. 

Although our background estimate excludes a simple 
quasar-dominated model based on the observed number of 
such objects, the analysis here may give a background flux 
which is bias ed (to o large) from a combination of line blend- 
ing (section 3.3.2) and clustering around the background 
quasars. From the comparison with Williger et al. (1994), 
above, there is an upper limit on the correction for line 
blending, at the higher redshifts, of 0.7 dex. However, an 
analysis of the data when column density completeness lim- 
its were increased by Alog(JV) = 0.5 suggests that a change 
in the lower limits here of ~ 0.1 dex is more likely. A further 
change of up to between 0.5 and 1 dex is possi ble if quasars 
lie in regions of increased clustering (section 3.2.2). These 
two effects imply that at the highest redshifts the flux mea- 
sured here could reasonably overestimate the real value by 
~ 0.5 dex. This could make the measurements marginally 
consistent with the expected flux from the observed popu- 
lation of quasars. 

There is also some uncertainty in the expected back- 
ground from quasars since observations could be incomplete 
even at the better understood lower redshifts (eg. Gold- 
schmidt et al. 1992) and while absorption in damped sys- 
tems is understood in theory (Fall & Pei 1993) its effect 
is uncertain (particularly because the distribution of high 
column density systems is poorly constrained). 

The highest flux model (largest population of obscured 
quasars) from Fall & Pei (1995) is consistent with the mea- 
surements here (assuming that the objects used in this paper 
are not significantly obscured). 



6.4.2 Stars 

The background appears to be stronger than the integrated 
flux from the known quasar population. Can star formation 
at high redshifts explain the discrepancy? 

Recent results from observations of low redshift star- 
bursts (Leitherer et al. 1995) suggest that very few ionizing 
photons (< 3 %) escape from these systems. If high redshift 
starbursts are similar in their properties, then the presence 
of cool gas in these objects would similarly limit their contri- 
bution to the ionizing background. However, Madau & Shull 
(1996) estimate that if star formation occurs in Lyman-a 
clouds, and a significant fraction of the ionizing photons 



(~ 25 %) escape, then these photons may contribute a sub- 
stantial fraction of the ionizing background photons in their 
immediate vicinity. As an example, at z ~ 3 they estimate 
that J„ < 50 J23 if star formation sets in at z ~ 3.2. This flux 
would dominate the lowest (no correction for obscuration) 
quasar background shown in figure [l2] and could be consis- 
tent with the intensity we estimate for the background at 
this redshift, given t he po ssible systematic biases discussed 
above and in section 3.2.2. 



7 CONCLUSIONS 

A model has been fitted to the population of Lyman-a 
clouds. The model includes the relative effect of the ionising 
flux from the background and nearby quasars (section |^) . 

The derived model parameters for the population of ab- 
sorbers are generally consistent with earlier estimates. There 
is some evidence that /3, the column density power law pop- 
ulation exponent, increases with col umn density, but could 
also be due to line blending (section 3.1). 



The ionising background is estimated to be 100 _g J23 
(model B, section [a.2| ) over the range of redshifts (2 < z < 
4.5) covered by the data. No strong evidence for evolution in 
the ionizing background is seen over this redshift range. In 
particular, th ere is no significant evidence for a decline for 
z > 3 (section [6.2.3 ). Previous results may have been biased 



6.2.4) 



(too low, with optimistic error limits — section 

Constraints on the evolution of the background are 
shown in figure The estimates are not consistent with 
the background flu x ex pected from the observed population 
of quasars (section |6.4| ). However, two effects are likely to be 
important. First, both line blending and increased cluster- 
ing of clouds near quasars lead to the measured background 
being overestimated. Second, a significant fraction of the 
quasar population at high redshifts may be obscured. Since 
their contribution to the background would then be underes- 
timated this would imply that current models of the ionizing 
background are too low. Both of these would bring the ex- 
pected and measured fluxes into closer agreement. It is also 
possible that gravitational lensing makes the measurement 
here an overestimate of the true background. 

The dominant source of errors in our work is the lim- 
ited number of lines near the background quasar (eg. fig- 
ures § and 0). Systematic errors are smaller and become 
important only if it is necessary to make standard (unob- 
scured quasar) models for the background consistent with 
the lower limits presented here. Further data will therefore 
make the estimate here more accurate, although observa- 
tional data are limited by confusion of the most numerous 
lower column density systems (log(JV) < 13.0) so it will re- 
main difficult to remove the bias from line blending. An im- 
provement in the errors for the highest redshift data points, 
or a determination of the shape of the ionizing spectrum 
(e.g. from He II/H I estimates in Lyman-a clouds) would 
help in discriminating between current competing models 
for the ionizing background. Finally, a determination of the 
background strength in the redshift range 0.5 < z < 2.0 is 
still needed. 
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